Radiology Reimagined: Advancing Clinical Practice Through AI Innovation (2025)-"Guiding the Future of AI Science in Radiology"

"Guiding the Future of AI Science in Radiology" – Dr. Charles Kahn

Pdf Summary

This presentation by Dr. Charles E. Kahn Jr. from the University of Pennsylvania discusses guiding the future of AI science in radiology, focusing on how to judge AI research quality, identify common problems, and understand performance metrics. Key points include:<br /><br />1. Hypothesis and Data: Emphasizes clarity in hypotheses—quantitative vs. qualitative, retrospective vs. prospective. Data must be de-identified, ethically managed, appropriately prepared (normalization, resampling), and representative of the intended use. Ground truth annotation should be well-defined with qualified annotators and processes to handle variability.<br /><br />2. Model and Evaluation: Discusses architectures used in AI radiology like ResNet, U-Net, and others. Proper model training involves consideration of hyperparameters and transfer learning. Evaluation metrics must measure relevant outcomes and include metrics like true positive rate, ROC curves, calibration curves, confusion matrices, and sensitivity analyses. The importance of external testing and statistical rigor is stressed.<br /><br />3. Data Handling: Data partitioning into training (70%), tuning/validation (20%), and testing (10%) sets should be strictly disjoint by patient or institution to avoid data leakage, a serious methodological concern.<br /><br />4. Reporting and Guidelines: Adoption of standardized reporting guidelines for AI studies (CLAIM, STARD, QUADAS-AI, MI-CLAIM-LLM, GENAISIS) is encouraged for transparency and reproducibility.<br /><br />5. Performance Metrics and Calibration: The talk highlights the importance of selecting appropriate metrics (classification, segmentation, calibration) and acknowledges the existence of 191 metrics with many synonyms detailed in the ROADMAP ontology.<br /><br />6. Future Directions: Urges the AI research community to improve methodological rigor to catch issues such as data leakage before publication, improve overall scientific quality, and foster innovation.<br /><br />Participants are invited to further engage with RSNA resources to promote best practices in AI research within radiology.

Keywords

AI in radiology

Hypothesis clarity

Data de-identification

Ground truth annotation

ResNet

U-Net

Model evaluation metrics

Data leakage prevention

Standardized reporting guidelines

Methodological rigor in AI research